Natural Compression for Distributed Deep Learning

Modern deep learning models are often trained in parallel over a collection of distributed machines to reduce training time. In such settings, communication of model updates among machines becomes a significant performance bottleneck and various lossy update compression techniques have been proposed to alleviate this problem. In this work, we introduce a new, simple yet theoretically and practically effective compression technique: {\em natural compression (NC)}. Our technique is applied individually to all entries of the to-be-compressed update vector and works by randomized rounding to the nearest (negative or positive) power of two, which can be computed in a "natural" way by ignoring the mantissa. We show that compared to no compression, NC increases the second moment of the compressed vector by not more than the tiny factor \nicefrac{9}{8}, which means that the effect of NC on the convergence speed of popular training algorithms, such as distributed SGD, is negligible. However, the communications savings enabled by NC are substantial, leading to {\em $3$-$4\times$ improvement in overall theoretical running time}. For applications requiring more aggressive compression, we generalize NC to {\em natural dithering}, which we prove is {\em exponentially better} than the common random dithering technique. Our compression operators can be used on their own or in combination with existing operators for a more aggressive combined effect, and offer new state-of-the-art both in theory and practice.Comment: 8 pages, 20 pages of Appendix, 6 Tables, 14 Figure

A higher-order active contour model of a `gas of circles' and its application to tree crown extraction

Many image processing problems involve identifying the region in the image domain occupied by a given entity in the scene. Automatic solution of these problems requires models that incorporate significant prior knowledge about the shape of the region. Many methods for including such knowledge run into difficulties when the topology of the region is unknown a priori, for example when the entity is composed of an unknown number of similar objects. Higher-order active contours (HOACs) represent one method for the modelling of non-trivial prior knowledge about shape without necessarily constraining region topology, via the inclusion of non-local interactions between region boundary points in the energy defining the model. The case of an unknown number of circular objects arises in a number of domains, e.g. medical, biological, nanotechnological, and remote sensing imagery. Regions composed of an a priori unknown number of circles may be referred to as a `gas of circles'. In this report, we present a HOAC model of a `gas of circles'. In order to guarantee stable circles, we conduct a stability analysis via a functional Taylor expansion of the HOAC energy around a circular shape. This analysis fixes one of the model parameters in terms of the others and constrains the rest. In conjunction with a suitable likelihood energy, we apply the model to the extraction of tree crowns from aerial imagery, and show that the new model outperforms other techniques

A flexible software architecture concept for the creation of accessible PDF documents

This paper presents a flexible software architecture concept that allows the automatic generation of fully accessible PDF documents originating from various authoring tools such as Adobe InDesign or Microsoft Word. The architecture can be extended to include any authoring tools capable of creating PDF documents. For each authoring tool, a software accessibility plug-in must be implemented which analyzes the logical structure of the document and creates an XML representation of it. This XML file is used in combination with an untagged non-accessible PDF to create an accessible PDF version of the document. The implemented accessibility plug-in prototype allows authors of documents to check for accessibility issues while creating their documents and add the additional semantic information needed to generate a fully accessible PDF document

Cosmology with Gamma-Ray Bursts Using k-correction

In the case of Gamma-Ray Bursts with measured redshift, we can calculate the k-correction to get the fluence and energy that were actually produced in the comoving system of the GRB. To achieve this we have to use well-fitted parameters of a GRB spectrum, available in the GCN database. The output of the calculations is the comoving isotropic energy E_iso, but this is not the endpoint: this data can be useful for estimating the {\Omega}M parameter of the Universe and for making a GRB Hubble diagram using Amati's relation.Comment: 4 pages, 6 figures. Presented as a talk on the conference '7th INTEGRAL/BART Workshop 14 -18 April 2010, Karlovy Vary, Czech Republic'. Published in Acta Polytechnic

Fast R Functions for Robust Correlations and Hierarchical Clustering

Many high-throughput biological data analyses require the calculation of large correlation matrices and/or clustering of a large number of objects. The standard R function for calculating Pearson correlation can handle calculations without missing values efficiently, but is inefficient when applied to data sets with a relatively small number of missing data. We present an implementation of Pearson correlation calculation that can lead to substantial speedup on data with relatively small number of missing entries. Further, we parallelize all calculations and thus achieve further speedup on systems where parallel processing is available. A robust correlation measure, the biweight midcorrelation, is implemented in a similar manner and provides comparable speed. The functions cor and bicor for fast Pearson and biweight midcorrelation, respectively, are part of the updated, freely available R package WGCNA. The hierarchical clustering algorithm implemented in R function hclust is an order n3 (n is the number of clustered objects) version of a publicly available clustering algorithm (Murtagh 2012). We present the package flashClust that implements the original algorithm which in practice achieves order approximately n2, leading to substantial time savings when clustering large data sets

A practical review on the measurement tools for cellular adhesion force

Cell cell and cell matrix adhesions are fundamental in all multicellular organisms. They play a key role in cellular growth, differentiation, pattern formation and migration. Cell-cell adhesion is substantial in the immune response, pathogen host interactions, and tumor development. The success of tissue engineering and stem cell implantations strongly depends on the fine control of live cell adhesion on the surface of natural or biomimetic scaffolds. Therefore, the quantitative and precise measurement of the adhesion strength of living cells is critical, not only in basic research but in modern technologies, too. Several techniques have been developed or are under development to quantify cell adhesion. All of them have their pros and cons, which has to be carefully considered before the experiments and interpretation of the recorded data. Current review provides a guide to choose the appropriate technique to answer a specific biological question or to complete a biomedical test by measuring cell adhesion